• Computers evolved from people to mechanical calculators to silicon-based machines
• Modern computers use the Von Neumann architecture, storing both instructions and data in memory
• Computers represent data using binary (base 2) numbers made up of 0s and 1s
• A bit is a single binary digit; 8 bits make a byte
• Hexadecimal (base 16) is a compact way to represent binary, with each hex digit corresponding to 4 bits
• Abstraction allows representing complex data like images and text using numbers

• Images can be broken down into a grid of coloured pixels
• Colours are represented using the RGB model, with each colour channel (red, green, blue) ranging from 0-255
• 8-bit color uses 256 levels per channel, allowing for over 16 million possible colors
• Text is broken into individual characters, with each character mapped to a number using an encoding like ASCII
• ASCII is a simple lookup table mapping the numbers 0-255 to characters
• Unicode extends ASCII to support accented characters and symbols from all languages

• Konrad Zuse created the first programmable computers and high-level programming language in the 1940s
• Assembly allows writing human-readable instructions that map closely to machine code
• High-level languages like Python abstract away hardware details and are more portable across systems
• Low-level languages are harder to read and write but very fast and efficient
• Compiled languages are converted to machine code before execution; interpreted languages are executed on the fly

• Today, we will learn about the command line, a text-based interface to interact with computers
• We will learn some basic commands to navigate the file system, create and delete files, and run programs
• We will also learn about shell scripting, a way to automate tasks using the command line
• The command line is still widely used in data science and programming, especially for remote servers, cloud computing, and automation

Credit Dave Kerr

• The operating system (OS) is system software that interfaces with (and manages access to) a computer’s hardware. It also provides software resources
• The OS is divided into the kernel and user space
• The kernel is the core of the OS. It’s responsible for interfacing with hardware (drivers), managing resources etc. Running software in the kernel is extremely sensitive! That’s why users are kept away from it!
• The user space provides an interface for users, who can run programs/applications on the machine. Hardware access of programmes (e.g., memory usage) is managed by the kernel. Programmes in user space are essentially in sandboxes, which sets a limit to how much damage they can do.

Credit Dave Kerr/Kkchaudhary11

• The shell is just a general name for any user space program that allows access to resources in the system, via some kind of interface
• Shells come in many different flavours but are generally provided to aid a human operator in accessing the system. This could be interactively, by typing at a terminal, or via scripts, which are files that contain a sequence of commands
• Modern computers use graphical user interfaces (GUIs) as the standard tool for human-computer interaction
• Why “kernel” and “shell”? The kernel is the soft, edible part of a nut or seed, which is surrounded by a shell to protect it. Useful metaphor, innit?

Credit Dave Kerr

• Things are still a bit more complicated
• We’re not directly interacting with the “shell” but using a terminal
• A terminal is just a program that reads input from the keyboard, passes that input to another programme, and displays the results on the screen
• A shell program on its own does not do this - it requires a terminal as an interface
• Why “terminal”? Back in the old days (before computer screen existed), terminal machines (hardware!) were used to let humans interface with large machines (“mainframes”). Often many terminals were connected to a single machine
• When you want to work with a computer in a data center (or remotely in cloud computing), you’ll still do pretty much the same

Credit Dave Kerr

• Terminals are really quite simple - they’re just interfaces

• The first thing that a terminal will do is run a shell - a programme we can use to operate the computer

• Back to the shell: the shell usually takes input

  • Interactively from the user via the terminal’s command line
  • Executes scripts (without command line)

• In interactive mode the shell then returns output

  • To the terminal where it is printed/shown
  • To files or other locations

• The command line represents what is shown and entered in the terminal. They can be customised (e.g., with colour highlighting) to make interaction more convenient

• It is important to note that there are many different shell programmes, and they differ in terms of functionality
• On most Unix-like systems, the default shell is a program called bash, which stands for “Bourne Again Shell”
• Other examples are the Z Shell (or zsh; default on MacOS), Windows Command Prompt (cmd.exe, the default CLI on MS Windows), Windows PowerShell, C Shell, and many more
• When a terminal opens, it will immediately start the user’s preferred shell programme. (This can be changed.)

Of course, it’s always possible to open up the shell directly if you prefer. It’s your turn!

• Linux

• Mac

• Windows

Feel free to check our class tutorial on how to set up your shell in VS Code.

Open your terminal and type the following commands (without the $):

{bash, eval=FALSE, echo=TRUE} echo $SHELL whoami pwd mkdir new-folder cd .. ls man ls # type 'j' to scroll down, 'k' to scroll up, 'q' to quit

Share your results with a colleague (or the class)!

All bash commands have the same basic syntax:

command option(s) argument(s)

Examples:

```{bash, eval=FALSE, echo=TRUE} # list files in the Documents directory # with human-readable sizes

ls -lh ~/Documents

<br>



```{bash, eval=FALSE, echo=TRUE}
# sort the file and remove duplicates

sort -u file.txt 

Commands

• You don’t always need options or arguments

• For example:

  • ls ~/Documents/ and ls -lh ~/Documents are both valid commands that will yield (different) output

• However, you always need a command.

All Bash commands have the same basic syntax:

command option(s) argument(s)

Examples:

```{bash, eval=FALSE, echo=TRUE} # list files in the Documents directory # with human-readable sizes

ls -lh ~/Documents

<br>



```{bash, eval=FALSE, echo=TRUE}
# sort the file and remove duplicates

sort -u file.txt 

Options (also called Flags)

• Start with a dash. Usually one letter.

• Multiple options can be chained under a single dash.

  {bash, eval=FALSE, echo=TRUE} ls -l -a -h /var/log # This works ls -lah /var/log # So does this

• An exception is with (rarer) options requiring two dashes.

  {bash, eval=FALSE, echo=TRUE} ls --group-directories-first --human-readable /var/log

• l: Use a long listing format. This option shows detailed information about the files and directories

• h: With -l, print sizes in human-readable format (e.g., KB, MB)

• u: Unique, it filters out the duplicate entries in the output

• Think it’s difficult to memorize what the individual letters stand for? You’re totally right!

All Bash commands have the same basic syntax:

command option(s) argument(s)

Examples:

{bash, eval=FALSE, echo=TRUE} $ ls -lh ~/Documents/

{bash, eval=FALSE, echo=TRUE} $ sort -u file.txt

Arguments

• Tell the command what to operate on.

• Totally depends on the command what legit inputs are.

• Can be a file, path, a set of files and folders, a string, and more

• Sometimes more than just one argument is needed:

  {bash, eval=FALSE, echo=TRUE} mv figs/cat.png best-figs/cat02.png

• The man tool can be used to look at the manual page for a topic.

• The man pages are grouped into sections, we can see them with man man.

• The cht.sh website can be used directly from the shell to get help on tools. Run it like this: curl cht.sh/command

• You can also install the tldr tool which provides simplified help pages for common commands. Run it like this: tldr command

{bash, echo=TRUE} tldr ls

• For more info on how to get help, see here.

Manual pages are shown in the shell. Here are the essentials to navigate through contents presented in the pager:

• d - Scroll down half a page
• u - Scroll up half a page
• j / k - Scroll down or up a line. You can also use the arrow keys for this
• q - Quit
• /pattern - Search for text provided as “pattern”
• n - When searching, find the next occurrence
• N - When searching, find the previous occurrence
• These and other man tricks are detailed in the help pages (hit “h” when you’re in the pager for an overview).

RTFM!

Always check the documentation!

Key navigation commands are: - pwd to print (the current) working directory. - cd to change directory.

{bash, echo=TRUE} pwd

You can use absolute paths, but it’s better to use relative paths and invoke special symbols for a user’s home folder (~), current directory (.), and parent directory (..) as needed

Beware of directory names that contain spaces. Say you have a directory called “My Documents”. (I’m looking at you, Windows.)

• Why won’t $ cd My Documents work?
• Bash syntax is super pedantic about spaces and ordering. Here it thinks that “My” and “Documents” are separate arguments.
• How to deal with it:
  • Use quotation marks: $ cd "My Documents".
  • Use Tab completion to automatically “escape” the space: $ cd My\ Documents.
  • Don’t use spaces in file and folder names. Just don’t.
  • Files and folders should be named in lowercase and use dashes or snake_case. These are useful conventions!
  • For instance: assignment-05 or assignment_05

• ls (list): Show files and directories in the current directory
• ls -l: Long listing format with detailed information
• ls -a: Show hidden files (those starting with a dot)
• ls -lh: Long listing format with human-readable sizes
• ls -R: List subdirectories recursively
• pwd (print working directory): Show the current directory path
• cd (change directory): Change the current working directory
• cd -: Go back to the previous directory
• .: Refers to the current directory
• ..: Refers to the parent directory
• ~: Refers to the home directory

• mkdir: Create a new directory
• touch: Create a new empty file or update timestamps
• cp: Copy files or directories
• mv: Move or rename files or directories
• rm: Remove files (use with caution!)
• rmdir: Remove empty directories
• cat: Display file contents
• find: Search for files and directories

For a more detailed overview, click here

• We will learn a bit more about the command line, especially about text processing and scripting
• We will also learn about how to use vim or neovim as a text editors
  • Vim is a powerful text editor that is highly configurable and can be used for many different programming languages. And it is my editor of choice! 🤓
• After that, we will introduce Git and GitHub for version control and collaboration